Reliability acquiring apparatus, reliability acquiring method, and reliability acquiring program

ABSTRACT

A reliability acquiring apparatus includes a section that stores information on a classifier that outputs part likelihood concerning a predetermined part of a detection target when applied to an image, a section that calculates, concerning an image region included in an input image, the part likelihood on the basis of the information on the classifier, a section that determines, on the basis of the calculated part likelihood, a position of the predetermined part in the input image, a section that stores information on a reference position of the predetermined part, a section that calculates, on the basis of the information on the reference position, difference information between the reference position of the predetermined part and the determined position of the predetermined part, and a section that calculates, on the basis of the difference information, reliability indicating possibility that the input image is an image of the detection target.

BACKGROUND

The present invention relates to a reliability acquiring apparatus, areliability acquiring method, and a reliability acquiring program.

A large number of detection techniques for specifying a position of atarget (e.g., a face region) in an image have been proposed. In most ofsuch detection techniques, a target is detected when an input image isgiven, by thoroughly searching through the input image using aclassifier that determines whether an image region is the target.

The classifier used in face detection is usually generated by preparinga large quantity of cropped images of face regions and images notincluding a face and performing learning. However, it is difficult toprepare an image group for the learning completely including informationnecessary for discriminating whether a face region is a face. Therefore,in general, a built classifier involves a certain degree of detectionerrors. The detection errors include two kinds of detection errors: aface region cannot be determined as the face and is overlooked(un-detection); and a region that is not the face is determined as theface by mistake (misdetection). Concerning the latter, severaltechniques for reducing the number of misdetections are known.

For example, Patent Document 1 describes, as the method of reducingmisdetections of a face, a method of determining whether a regiondetected as a face is the face according to whether a color of theregion is a skin color.

Patent Document 2 describes, as the method of reducing misdetections ofa face, a method of determining whether a face region is a face using astatistical model concerning textures and shapes of faces. In thismethod, parameters of the statistical model is adjusted and a differencein intensity values between a face image generated from the model and ancropped image of a face region on the basis of a face detection resultis minimized. Note that the face image generated from the model and thecropped image of the face region on the basis of the face detectionresult are respectively normalized concerning a face shape. When theminimized difference in the intensity values is equal to or larger thana predetermined threshold, it is determined that a detection result is amisdetection. Usually, the statistical model concerning the face haspoor expression of images other than the face, if the cropped face imageon the basis of the face detection result is not the face(misdetection), the difference in the intensity values between the faceimage generated from the model and the cropped face image on the basisof the face detection result is considered to increase. The methoddescribed in Patent Document 2 is a method of determining on the basisof such knowledge whether the face detection result is truly the faceaccording to the difference in the intensity values.

Non-Patent Document 1 proposes, as the method of reducing misdetectionsof a face, a method of learning a face misdetection determination deviceusing a support vector machine (SVM) and applying the determinationdevice to a region cropped on the basis of a face detection result tomore actively eliminate a misdetection. In this method, a classifier isbuilt that learns an image feature value extracted by Gabor wavelettransformation using the SVM and identifies whether a texture of atarget region is like a face.

Patent Document 1: Patent Publication JP-A-2009-123081

Patent Document 2: Patent Publication JP-A-2010-191592

Non-Patent Document 1: Yamashita, et al., “False Detection Reduction forFace Detection”, The Institute of Electronics, Information andCommunication Engineers Technical Research Report PRMU2004-102

Non-Patent Document 2: D. Cristinacce and T. F. Cootes, “A Comparison ofShape Constrained Facial Feature Detectors,” In 6th InternationalConference on Automatic Face and Gesture Recognition 2004, Korea, pages357-380, 2004

SUMMARY

However, in the method of Patent Document 1, it is determined whetherthe region detected as the face is the face on the basis of whether thecolor in the region detected as the face is the skin color. Therefore,there is a problem in that, whereas a background other than the skincolor can be eliminated as a region other than the face, when a regionhaving a color close to the skin color is included in the background orwhen a color of the skin of the face changes because of the influence ofillumination fluctuation, it is impossible to appropriately determinedwhether the region detected as the face is the face.

In the method of Patent Document 2, the difference concerning theintensity image between the detected face region and the statisticalmodel concerning the face is used. In this case, in an environment inwhich illumination, facial expression, and the like complicatedlychange, intensity values in a face region also complicatedly changes.Therefore, it is difficult to fixedly determine at which degree of adifference value between the detected face region and the intensityimage of the statistical model a misdetection occurs. Therefore, themethod of determining whether the region detected as the face is theface on the basis of the difference value of the intensity image has aproblem in that accuracy is not sufficiently obtained.

Further, in the method of Non-Patent Document 1, the image feature valueby the Gabor wavelet transformation is extracted from the entireextracted face region and is used for determining whether the faceregion is the face. However, when a face detector determines a region,which is not a face, such as a background as the face by mistake, in thefirst place, the misdetection is considered to be caused by a face-liketexture included in the region. In this case, there is possibility thatthe image feature value extracted from the entire detected face regionindicates that the target region is like the face. Therefore, in themethod of determining whether the region detected as the face is theface according to the image feature value extracted from the entiredetected face region as in the method of Non-Patent Document 1, there isa problem in that accuracy is not sufficiently obtained and it isdifficult to reduce misdetections of the face.

These problems are problems that also occur when a part other than theface is a detection target.

Therefore, the present invention has been devised to solve the problemsand it is an object of the present invention to, in order to reducedetection errors such as misdetections, calculate, at high accuracy,reliability for determining whether an input image is an image of adetection target (e.g., a face image).

A reliability acquiring apparatus according to an aspect of the presentinvention includes: a classifier storing section that stores informationon a classifier that outputs part likelihood concerning a predeterminedpart of a detection target when applied to an image; a part likelihoodacquiring section that calculates, concerning an image region includedin an input image, the part likelihood on the basis of the informationon the classifier; a part position determining section that determines,on the basis of the calculated part likelihood, a position of thepredetermined part in the input image; a reference position informationstoring section that stores information on a reference position of thepredetermined part; a difference acquiring section that calculates, onthe basis of the information on the reference position, differenceinformation between the reference position of the predetermined part andthe determined position of the predetermined part; and a reliabilityacquiring section that calculates, on the basis of the differenceinformation, reliability indicating possibility that the input image isan image of the detection target.

In a reliability acquiring method according to an aspect of the presentinvention, a computer calculates, concerning an image region included inthe an input image and by referring to a classifier storing section thatstores information on a classifier that outputs part likelihoodconcerning a predetermined part of a detection target when applied to animage, the part likelihood on the basis of the information on theclassifier, determines a position of the predetermined part in the inputimage on the basis of the calculated part likelihood, calculates, byreferring to a reference position information storing section thatstores information on a reference position of the predetermined part andon the basis of the information on the reference position, differenceinformation between the reference position of the predetermined part andthe determined position of the predetermined part, and calculatespossibility that the input image is an image of the detection target onthe basis of the difference information.

A program according to an aspect of the present invention is a programfor causing a computer to realize: a function of calculating, concerningan image region included in the an input image and by referring to aclassifier storing section that stores information on a classifier thatoutputs part likelihood concerning a predetermined part of a detectiontarget when applied to an image, the part likelihood on the basis of theinformation on the classifier; a function of determining a position ofthe predetermined part in the input image on the basis of the calculatedpart likelihood; a function of calculating, by referring to a referenceposition information storing section that stores information on areference position of the predetermined part and on the basis of theinformation on the reference position, difference information betweenthe reference position of the predetermined part and the determinedposition of the predetermined part; and a function of calculatingpossibility that the input image is an image of the detection target onthe basis of the difference information.

Note that, in the present invention, “section” does not simply meansphysical means and also includes realization of a function of the“section” by software. A function of one “section” or device may berealized by two or more physical means or devices. Functions of two ormore “sections” or devices may be realized by one physical means ordevice.

According to the present invention, it is possible to calculate, at highaccuracy, reliability for determining whether an input image is an imageof a detection target.

DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a configuration example of a face regionreliability calculating apparatus 1 in an embodiment.

FIG. 2 is a flowchart for explaining the operation of the face regionreliability calculating apparatus 1 in the embodiment.

FIG. 3 is a diagram showing an example of an image input by an imageinput section.

FIG. 4 is a diagram showing facial feature points targeted by a facialfeature point reliability generating apparatus in an image showing aface.

FIG. 5 is a diagram showing an example of a reliability image of thepupil center of the right eye.

FIG. 6 is a diagram showing a position where reliability is maximized ina reliability image of the periphery of the pupil center of the righteye.

DETAILED DESCRIPTION

An embodiment of the present invention is explained below with referenceto the drawings in an example in which a detection target is a face andfeature point likelihood (facial feature point reliability) of featurepoints (facial feature points) corresponding to parts of the face suchas the eyes and the nose is used as part likelihood concerning the partsof the face.

FIG. 1 is a diagram showing a configuration example of a face regionreliability calculating apparatus 1 in an embodiment of the presentinvention. As shown in FIG. 1, the face region reliability calculatingapparatus 1 according to the present invention includes a dataprocessing device 100 and a storage device 200. The data processingdevice 100 includes a face image input section 110, a facial featurepoint reliability generating section 120, a facial feature pointposition determining section 130, a facial feature point positiondifference calculating section 140, and a face region reliabilitycalculating section 150. The storage device 200 includes a facialfeature point classifier storing section 210 and a face shape modelstoring section 220. Note that, although not shown in the figure, theface region reliability calculating apparatus 1 may include aconventional face detector.

The facial feature point classifier storing section 210 stores, for eachof facial feature points, information on a facial feature pointclassifier that outputs feature point likelihood when applied to animage. The facial feature point classifier can be generated usingvarious conventional techniques.

The face shape model storing section 220 stores, as information on areference position of parts of a face, information on a face shape modelfor specifying reference positions of facial feature points on the basisof statistical distribution concerning positions (coordinates) of thefacial feature points. As the face shape model, it is possible toassume, for example, a model for specifying, for each of the facialfeature points, as reference positions, average coordinates of facialfeature points of a plurality of people, a model for specifying,assuming a vector X having position coordinates of the facial featurepoints as elements, a subspace obtained by subjecting vectors X obtainedfrom a plurality of face images to principal component analysis, a modelfor specifying the reference positions of the facial feature pointsusing a parametrix function, and a model for retaining average positions(positions in an image coordinate system) of the facial feature pointsin an environment in which the size of an input image and aphotographing position of the face are fixed.

The face image input section 110 acquires an input image set as aprocessing target.

The facial feature point reliability generating section 120 calculates,concerning an image region (including pixels) included in the inputimage, facial feature point likelihood on the basis of the informationon the facial feature point classifier stored in the facial featurepoint classifier storing section 210 and generates a reliability imagerepresenting a distribution of the facial feature point likelihood.

The facial feature point position determining section 130 determines, onthe basis of the reliability image generated by the facial feature pointreliability generating section 120, positions of facial feature points(detected feature points) in the input image.

The facial feature point position difference calculating section 140calculates reference positions of the facial feature points on the basisof the face shape model stored in the face shape model storing section220 and calculates difference information between the referencepositions of the facial feature points and the positions of the detectedfeature points determined by the facial feature point positiondetermining section 130.

The face region reliability calculating section 150 calculates, on thebasis of the difference information calculated by the facial featurepoint position difference calculating section 140, face regionreliability indicating possibility that the input image is a face image.

The operation of the face region reliability calculating apparatus 1 isexplained with reference to the drawings. FIG. 2 is a flowchart forexplaining the operation of the face region reliability calculatingapparatus 1 shown in FIG. 1.

First, the face image input section 110 acquires an input image set as aprocessing target (step S111).

Subsequently, the facial feature point reliability generating section120 calculates, concerning an image region included in the input image,facial feature point likelihood on the basis of the information on thefacial feature point classifier stored in the facial feature pointclassifier storing section 210 and generates a reliability imagerepresenting a distribution of the facial feature point likelihood (stepS112).

Subsequently, the facial feature point position determining section 130determines, on the basis of the reliability image generated in S112,positions of facial feature points (detected feature points) in theinput image (step S113).

Subsequently, the facial feature point position difference calculatingsection 140 calculates reference positions of the facial feature pointson the basis of the face shape model stored in the face shape modelstoring section 220 and calculates difference information between thereference positions of the facial feature points and the positions ofthe detected feature points determined in S113 (step S114).

Subsequently, the face region reliability calculating section 150calculates, on the basis of the difference information calculated inS114, face region reliability indicating possibility that the inputimage is a face image (step S115).

According to this embodiment, the face region reliability calculatingsection calculates face region reliability on the basis of whetherpositions of facial feature points of parts such as the eyes, the nose,and the mouth are arranged to be like a face. Therefore, it is possibleto calculate face region reliability at higher practical accuracy. Byusing such face region reliability, even when face detector detects aregion, which is not the face, such as a background as a face region bymistake, it is possible to accurately determine whether the detectedface region is truly the face.

The configuration and the operation in the embodiment of the presentinvention are explained with reference to a specific example.

In the face region reliability calculating apparatus 1 according to thepresent invention shown in FIG. 1, the data processing device 100 can beconfigured using an information processing apparatus such as a personalcomputer or a portable information terminal. The storage device 200 (thefacial feature point classifier storing section 210 and the face shapemodel storing section 220) can be configured by, for example, asemiconductor memory or a hard disk.

The face image input section 110, the facial feature point reliabilitygenerating section 120, the facial feature point position determiningsection 130, the facial feature point position difference calculatingsection 140, and the face region reliability calculating section 150 canbe realized by, in the data processing device 100, for example, a CPU(central processing unit) executing a program stored in a storingsection. Note that a part or all of the sections of the data processingdevice 100 may be realized as hardware. For example, the face imageinput section 110 may include an imaging section such as a digitalcamera or a scanner and may include a communication module or the likefor communicating with an external device and acquiring an input image.

The face image input section 110 acquires an input image for which faceregion reliability is calculated. The input image to be acquired may bea face region image detected by a conventional face detector or may bean image obtained by imaging a person using a digital camera or thelike.

FIG. 3 is a diagram showing an example of the input image. The inputimage may include a background other than a face. When the face regionreliability calculating apparatus 1 includes a face detector, the faceregion reliability calculating apparatus 1 can apply face detectionprocessing to the image imaged by the digital camera or the like, crop aface region image, and use the face region image as the input image.

The facial feature point reliability generating section 120 applies thefacial feature point classifier, which is stored in the facial featurepoint classifier storing section 210, to the input image acquired by theface image input section 110 and calculates, for example, concerningpixels of the input image, feature point likelihood corresponding toparts of the face such as the eyes and the nose.

FIG. 4 is a diagram showing an example of the facial feature points. InFIG. 4, the facial feature points are indicated by X marks. In thisexample, as shown in FIG. 4, fourteen points, i.e., both the ends of theleft and right eyebrows, the centers and both the ends of the left andright eyes, a lower part of the nose, and both the ends and the centerof the mouth are used as the facial feature points. Note that the facialfeature points are not limited to the example shown in FIG. 4. Besidesthe fourteen points, the facial feature points may be one point.

The facial feature point reliability generating section 120 generates,for each of the facial feature points, a reliability image indicatingfacial feature point likelihood as a pixel value. In the example shownin FIG. 4, fourteen reliability images are generated. Note that, as amethod of applying the facial feature point classifier to calculate thefacial feature point likelihood, conventionally proposed various methodscan be used. For example, as described in Patent Literature 2, aclassifier for each of facial feature points configured using AdaBoostbased on Haar-like features by Viola and Jones is applied to an entireregion of an input image to generate a reliability image. In this way,in learning of the facial feature point classifier, by using imagefeature values and a learning algorithm robust against illumination andfacial expression fluctuation, it is possible to calculate face regionreliability robust against illumination and facial expressionfluctuation.

FIG. 5 is a diagram showing an example of a reliability imagecorresponding to the right eye center. In the example shown in FIG. 5,as facial feature point likelihood is larger, the reliability image isshown as denser black. It is indicated that, besides the right eyecenter, the facial feature likelihood in the right eye center is large(likely to be the right eye center) near the pupil center of the lefteye, near the right eyebrow, and near the nose bottom.

The facial feature point position determining section 130 determines, onthe basis of the reliability images generated by the facial featurepoint reliability generating section 120, positions of the facialfeature points in the input image. The facial feature point positiondetermining section 130 can determine, as the corresponding positions ofthe facial feature points, positions where facial feature valuelikelihood is maximized in the reliability images generated by thefacial feature point reliability generating section 120. Alternatively,instead of the positions where the facial feature point likelihood ismaximized in the reliability images, the facial feature point positiondetermining section 130 may determine, as the positions of the facialfeature points, positions where a product of a prior distribution offacial feature point positions and the facial feature point likelihoodis maximized.

FIG. 6 is a diagram in which a position where the facial feature pointlikelihood is maximized is indicated by an X mark in the reliabilityimage corresponding to the right eye center.

The facial feature point position difference calculating section 140calculates, on the basis of the face shape model stored in the faceshape model storing section 220, positions of reference positions (faceshape model feature points) of the facial feature points determined onthe basis of the face shape model and calculates, for each of the facialfeature points, difference information between the reference positionsof the facial feature points and the facial feature points (detectedfeature points) in the input image determined by the facial featurepoint position determining section 130.

The calculation of the difference information of the facial featurepoint positions is performed, for example, as explained below. Notethat, in the following explanation, it is assumed that the referencepositions of the facial feature points are directly recorded in the faceshape model storing section 220 as the face shape model. Specifically,in this example, as the face shape model, two-dimensional coordinatevalues (twenty-eight values) are respectively recorded concerning thefourteen facial feature points shown in FIG. 4. Similarly, the positionsof the detected feature points determined by the facial feature pointposition determining section 130 are two-dimensional coordinate values(twenty-eight values) concerning the fourteen facial feature pointsshown in FIG. 4.

There is a difference between a coordinate system of the face shapemodel feature points and a coordinate system of the detected featurepoints. Therefore, to calculate a difference in positions among thesefeature points, both the coordinate systems need to be aligned.Therefore, first, coordinate transformation p from a coordinate t of thedetected feature point to a coordinate k of the face shape model featurepoint is calculated.

In this example, as the coordinate transformation p, Helmerttransformation, which is coordinate transformation concerning an x-axisdirection, a y-axis direction, rotation in an in-screen direction, and ascale, is used. In this case, the coordinate transformation p isspecified by four parameters (pa, pb, pc, and pd) for specifying theHelmert transformation indicated by Expression (1). In Expression (1), tindicates a coordinate before the transformation and u indicates acoordinate after the transformation.

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 1} \rbrack & \; \\{\begin{pmatrix}u_{x} \\u_{y}\end{pmatrix} = {{\begin{pmatrix}p_{a} & p_{b} \\{- p_{b}} & p_{a}\end{pmatrix}\begin{pmatrix}t_{x} \\t_{y}\end{pmatrix}} + \begin{pmatrix}p_{c} \\p_{d}\end{pmatrix}}} & (1)\end{matrix}$

In this example, a parameter of the Helmert transformation p iscalculated by a least square method. In this case, p that minimizes asquare error (Expression (2)) from a face shape model feature point whenthe detected feature point t is transformed by certain coordinatetransformation p is a parameter of the Helmert transformation thatshould be calculated. Note that N in Expression (2) represents thenumber of facial feature points. When square errors are calculatedconcerning the fourteen facial feature points shown in FIG. 4, N=14.

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 2} \rbrack & \; \\{\sum\limits_{i = 1}^{N}{{{p( t_{i} )} - k_{i}}}^{2}} & (2)\end{matrix}$

The coordinate transformation p for minimizing the square errorrepresented by Expression (2) can be analytically calculated accordingto Expression (3). Note that n in Expression (3) is the number of datain calculating a least square and [z] is an average of z.

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 3} \rbrack & \; \\{{{p_{a} = \frac{{\lbrack t_{x} \rbrack\lbrack u_{x} \rbrack} + {\lbrack t_{y} \rbrack\lbrack u_{y} \rbrack} - {n\lbrack {{t_{x}u_{x}} + {t_{y}u_{y}}} \rbrack}}{\lbrack t_{x} \rbrack^{2} + \lbrack t_{y} \rbrack^{2} - {n\lbrack {t_{x}^{2} + t_{y}^{2}} \rbrack}}}{p_{b} = \frac{{\lbrack t_{y} \rbrack\lbrack u_{x} \rbrack} - {\lbrack t_{x} \rbrack\lbrack u_{y} \rbrack} - {n\lbrack {{t_{y}u_{x}} - {t_{x}u_{y}}} \rbrack}}{\lbrack t_{x} \rbrack^{2} + \lbrack t_{y} \rbrack^{2} - {n\lbrack {t_{x}^{2} + t_{y}^{2}} \rbrack}}}p_{c} = {\frac{1}{n}\{ {\lbrack u_{x} \rbrack - {p_{a}\lbrack t_{x} \rbrack} - {b\lbrack t_{y} \rbrack}} \}}}{p_{d} = {\frac{1}{n}\{ {\lbrack u_{y} \rbrack - {p_{a}\lbrack t_{y} \rbrack} - {b\lbrack t_{x} \rbrack}} \}}}} & (3)\end{matrix}$

Subsequently, a Euclidian distance between the coordinate k of the faceshape model feature point and the coordinate t of the detected featurepoint is calculated for each of the facial feature points as adifference ε according to Expression (4), using the calculatedcoordinate transformation p.

[Math 4]ε_(i) =∥p(t _(i))−k _(i)∥  (4)

Note that, in calculating the difference ε between the coordinate k ofthe face shape model feature point and the coordinate t of the detectedfeature point, the facial feature point position difference calculatingsection 140 may use another distance scale such as a Mahalanobisdistance rather than the Euclidian distance between the coordinate k ofthe face shape model feature point and the coordinate t of the detectedfeature point.

Further, when the facial feature point position determining section 130fails in determining a position of a facial feature point (e.g., when awrong position is determined as a facial feature point in a situation inwhich a facial feature point is blocked by sunglasses or a mask or asituation in which an input image is unclear and it is difficult tospecify a position of a facial feature point), the facial feature pointposition difference calculating section 140 may treat the facial featurepoint as an outlier and process the facial feature point. Specifically,a method of not using, in calculating the coordinate transformation pfrom the detected feature point t to the face shape model feature pointk, a facial feature point where facial feature point likelihood of aposition determined by the facial feature point position determiningsection 130 is equal to or smaller than a predetermined threshold isconceivable. To calculate a difference of a position of a facial featurepoint taking into account a facial feature point, a position of whichdetermined by the facial feature point position determining section 130greatly deviates from a true position (e.g., when a facial feature pointof the right eye is determined as being present in a position near theleft eye), the coordinate transformation p from the coordinate k of theface shape model feature point to the coordinate t of the detectedfeature point can be calculated by a method of robust estimation. As therobust estimation, conventionally proposed various methods can be used.

As an example of a method of calculating the coordinate transformation pusing the robust estimation in the facial feature point positiondifference calculating section 140, a method of calculating Helmerttransformation p from the coordinate t of the detected feature point tothe coordinate k of the face shape model feature point using a leastmedian of squares (LMedS) method is explained.

First, two facial feature points are selected at random from thefourteen facial feature points shown in FIG. 4. In the followingexplanation, the facial feature points selected at random are indicatedby signs “a” and “b”. Among the detected feature points, a set ofcoordinates corresponding to the two facial feature points selected atrandom is represented as (ta, tb). Among the face shape model featurepoints, a set of coordinates corresponding to the two facial featurepoints selected at random is represented as (ka, kb). Note that ka, kb,ta, and tb are respectively two-dimensional vectors representingcoordinate values.

Subsequently, a parameter of the Helmert transformation p from the setof coordinates (ta, tb) to the set of coordinates (ka, kb) iscalculated. In this case, since the transformation is from two points totwo points, the parameter is uniquely calculated.

Subsequently, the fourteen points of the coordinate t of the detectedfeature points are subjected to coordinate transformation according tothe calculated Helmert transformation p. A converted coordinate isrepresented as u. Subsequently, a Euclidian distance between thecoordinate u and the coordinate k is calculated for each of the fourteenfacial feature points. A median of the distances for the fourteen pointsis retained.

The above processing is repeated. The Helmert transformation p havingthe smallest Euclidian distance is finally adopted.

The face region reliability calculating section 150 calculates, on thebasis of the difference information of the position for each of thefacial feature points calculated by the facial feature point positiondifference calculating section 140, face region reliability J indicatingpossibility that the input image is a face image and stores the faceregion reliability J in the storing section. The face region reliabilityJ stored in the storing section can be read out by various applicationssuch as face recognition and can be used according to purposes of theapplications.

The calculation of the face region reliability J can be performed bycalculating, from the difference ε of the position of the facial featurepoint calculated by the facial feature point position differencecalculating section 140, according to Expression (5), a median of avalue obtained by converting the difference ε with a function. Thefunction σ is a function, a value of which decreases when a value of thedifference ε increases. For example, a sigmoid function indicated byExpression (6) is used. “a” and “b” in Expression (6) are parameters foradjusting a degree of a reduction of the value of the function σ whenthe value of the difference ε increases. “a” is a negative number.

$\begin{matrix}\lbrack {{Math}\mspace{14mu} 5} \rbrack & \; \\{J = {{med}( {\sigma( ɛ_{i} )} )}} & (5) \\\lbrack {{Math}\mspace{14mu} 6} \rbrack & \; \\{{\sigma(ɛ)} = \frac{1}{1 + {\exp( {- {a( {ɛ - b} )}} )}}} & (6)\end{matrix}$

Note that the face region reliability calculating section 150 maycalculate an average of a value obtained by converting the difference εwith the function σ as the face region reliability J rather thancalculating the median of the value obtained by converting thedifference ε of the position of the facial feature point with thefunction σ as the face region reliability J.

In calculating the face region reliability J, the face regionreliability calculating section 150 may use the facial feature pointlikelihood in the position of the detected feature point calculated bythe facial feature point position determining section 130 in addition tothe difference ε of the position of each of the facial feature pointcalculated by the facial feature point position difference calculatingsection 140.

In this case, the calculation of the face region reliability J can beperformed by calculating, when the difference of the position of each ofthe facial feature points is represented as ε and facial feature pointlikelihood of each of the facial feature points is represented as s, aweighted sum of a median (or an average) of a value obtained byconverting the difference ε with the function σ and a median (anaverage) of the facial feature likelihood s according to Expression (7).In Expression (7), c is a parameter for adjusting a balance of thedifference ε of the facial feature point position and the facial featurepoint likelihood s. Note that c is a real number value in a range of 0to 1.

[Math 7]J=c×med(σ(ε_(i)))+(1−c)×med(s _(i))  (7)

In calculating the face region reliability J, the face regionreliability calculating section 150 may calculate integrated face regionreliability using one or a plurality of levels of additional face regionreliability (e.g., values representing face likelihood output by aconventional face detecting apparatus) calculated by a method differentfrom a method for calculating the face region reliability J, in additionto the difference ε of position of each of the facial feature pointscalculated by the facial feature point position difference calculatingsection 140 (or, in addition to that, the facial feature pointlikelihood s in the facial feature point position calculated by thefacial feature point position determining section 130).

For example, the face region reliability calculating section 150 maycalculate integrated face region reliability J that according toExpression (8) from the face region reliability J calculated accordingto Expression (5) from the difference ε of the position of each of thefacial feature points (or the face region reliability J calculatedaccording to Expression (7) from the difference ε and the facial featurepoint likelihood s of each of the facial feature points). In Expression.(8), d is a parameter for adjusting a balance of the face regionreliability J and an additional face region reliability J0. Note that dis a real number value in a range of 0 to 1.

[Math 8]Ĵ=d×J+(1−d)×J ₀  (8)

With the configuration in this example, the face region reliabilityacquiring section calculates face region reliability on the basis ofwhether positions of predetermined parts detected in an input image (inthis example, positions of facial feature points corresponding to theeyes, the nose, the mouth, and the like) are arranged like predeterminedparts of a detection target or, in addition to that, on the basis ofpart likelihood in detected positions (in this example, the facialfeature point likelihood s) rather than calculating reliabilityaccording to whether the input image as a whole includes a texture likea part of a detection target (in this example, the face). Therefore, itis possible to calculate face region reliability at higher practicalaccuracy. By using the face region reliability, even when the facedetector detects a region that is not the face such as a background as aface region, it is possible to accurately determine whether the detectedregion is truly the face.

The face region reliability calculating apparatus, the face regionreliability calculating method, and the face region reliabilitycalculating program according to this embodiment can be widely used forimprovement of accuracy in processing for receiving face images in facedetection, face authentication, facial expression recognition, and thelike as inputs.

Note that this embodiment is intended to facilitate understanding of thepresent invention and is not intended to limitedly interpret the presentinvention. The present invention can be changed and improved withoutdeparting from the spirit of the present invention. Equivalents of thepresent invention are also included in the present invention.

For example, in the example explained in the embodiment, the face regionreliability calculating apparatus calculates facial feature pointlikelihood as part likelihood. However, the present invention is notlimited to such a configuration. For example, a part other than the facecan be a detection target. As the part likelihood, region likelihood andthe like can be used rather than the feature point likelihood. That is,a classifier that outputs part likelihood of the detection target can begenerated. When a reference position of a part can be statisticallydetermined (a model can be generated), the present invention can beapplied to the detection target and the part to calculate reliability.

The present invention is explained above with reference to theembodiment. However, the present invention is not limited to theembodiment. Various changes that those skilled in the art can carry outwithin the scope of the present invention can be made to theconfiguration and the details of the present invention.

A part or all of the embodiment can be described as indicated by thefollowing notes but is not limited to the following.

-   (Note 1) A reliability acquiring apparatus including: a classifier    storing section that stores information on a classifier that outputs    part likelihood concerning a predetermined part of a detection    target when applied to an image; a part likelihood acquiring section    that calculates, concerning an image region included in an input    image, the part likelihood on the basis of the information on the    classifier; a part position determining section that determines, on    the basis of the calculated part likelihood, a position of the    predetermined part in the input image; a reference position    information storing section that stores information on a reference    position of the predetermined part; a difference acquiring section    that calculates, on the basis of the information on the reference    position, difference information between the reference position of    the predetermined part and the determined position of the    predetermined part; and a reliability acquiring section that    calculates, on the basis of the difference information, reliability    indicating possibility that the input image is an image of the    detection target.-   (Note 2) The reliability acquiring apparatus recited in note 1,    wherein the detection target is a face.-   (Note 3) The reliability acquiring apparatus recited in note 1 or 2,    wherein the classifier storing section and the reference position    information storing section respectively store, concerning a    plurality of predetermined parts, information on the classifier and    information on reference positions, and the part likelihood    acquiring section, the part position determining section, and the    difference acquiring section respectively, concerning the    predetermined parts, acquire the part likelihood, determine    positions of the predetermined parts, and calculate the difference    information.-   (Note 4) The reliability acquiring apparatus recited in any one of    notes 1 to 3, wherein the reliability acquiring section calculates    the reliability on the basis of the difference information and the    calculated part likelihood.-   (Note 5) The reliability acquiring apparatus recited in any one of    notes 1 to 4, wherein the reliability acquiring section calculates    integrated reliability on the basis of the reliability and one or    more levels of additional reliability calculated by a method    different from a method for calculating the reliability.-   (Note 6) The reliability acquiring apparatus recited in any one of    notes 1 to 5, wherein the difference acquiring section calculates    the difference information on the basis of whether the part position    determined by the position determining section is an outlier using a    method of robust estimation.-   (Note 7) A reliability acquiring method in which a computer    calculates, concerning an image region included in the an input    image and by referring to a classifier storing section that stores    information on a classifier that outputs part likelihood concerning    a predetermined part of a detection target when applied to an image,    the part likelihood on the basis of the information on the    classifier, determines a position of the predetermined part in the    input image on the basis of the calculated part likelihood,    calculates, by referring to a reference position information storing    section that stores information on a reference position of the    predetermined part and on the basis of the information on the    reference position, difference information between the reference    position of the predetermined part and the determined position of    the predetermined part, and calculates possibility that the input    image is an image of the detection target on the basis of the    difference information.-   (Note 8) A program for causing a computer to realize: a function of    calculating, concerning an image region included in the an input    image and by referring to a classifier storing section that stores    information on a classifier that outputs part likelihood concerning    a predetermined part of a detection target when applied to an image,    the part likelihood on the basis of the information on the    classifier; a function of determining a position of the    predetermined part in the input image on the basis of the calculated    part likelihood; a function of calculating, by referring to a    reference position information storing section that stores    information on a reference position of the predetermined part and on    the basis of the information on the reference position, difference    information between the reference position of the predetermined part    and the determined position of the predetermined part; and a    function of calculating possibility that the input image is an image    of the detection target on the basis of the difference information.

This application claims priority based on Japanese Patent ApplicationNo. 2012-31319 filed on Feb. 16, 2012, the entire disclosure of which isincorporated herein.

1 Face region reliability calculating apparatus

100 Data processing device

110 Face image input section

120 Facial feature point reliability generating section

130 Facial feature point position determining section

140 Facial feature point position difference calculating section

150 Face region reliability calculating section

200 Storage device

210 Facial feature point classifier storing section

220 Face shape model storing section

I claim:
 1. A reliability calculating apparatus for detection of atarget as a face, comprising: a processor; and a storing device, whereinthe storing device comprises: classifier storage that stores informationon a classifier that outputs a part likelihood of a predetermined partof a detection target which is calculated based on image feature valuesconcerning the predetermined part of the detection target, and referenceposition storage that stores information on a reference position of thepredetermined part, wherein the storing device further has, storedtherein, programming code that, upon execution by the processor, causesthe processor to perform the functions of: retrieving, from theclassifier storing device, information on the classifier from theclassifier storage, calculating, using data of an image region includedin an input image, a part likelihood of the input image by using theretrieved information on the classifier, determining, based on thecalculated part likelihood, a position of the predetermined part in theinput image, retrieving information on the reference position of thepredetermined part of the detection target stored from the referenceposition information storing, calculating difference information betweenthe reference position of the predetermined part of the detection targetand the determined position of the predetermined part, and calculatingfirst reliability indicating possibility that the input image is animage showing the detection target based on a facial feature pointlikelihood calculated based on the image feature values, wherein theimage feature values include a difference in intensity values, whereinthe programming code causes the processor, in calculating differenceinformation between the reference position of the predetermined part ofthe detection target and the determined position of the predeterminedpart, to calculate the difference information on the basis of whetherthe position of the predetermined part in the input image determined bythe position determining unit is an outlier using a method of robustestimation, and wherein the detection target is a face.
 2. Thereliability calculating apparatus according to claim 1, wherein theprogramming code further causes the processor to calculate a thirdreliability based on the first reliability and one or more secondreliabilities, and wherein each of the second reliabilities indicates apossibility that the input image is the image of the detection target,and is calculated in a manner different from that of calculating thefirst reliability.
 3. The reliability calculating apparatus according toclaim 1, wherein the classifier storage and the reference positioninformation storage respectively store information on part likelihoodand information on reference positions for a plurality of predeterminedparts, and the part likelihood acquiring unit, the part positiondetermining unit, and the difference acquiring unit, respectivelyconcerning the plurality of predetermined parts, acquire the partlikelihood, determine positions of the predetermined parts, andcalculate the difference information.
 4. The reliability calculatingapparatus according to claim 3, wherein the programming code, incalculating the first reliability, causes the processor to calculate thereliability on the basis of the difference information and thecalculated part likelihood.
 5. The reliability calculating apparatusaccording to claim 4, wherein the programming code further causes theprocessor to calculate a third reliability based on the firstreliability and one or more second reliabilities, and wherein each ofthe second reliabilities indicates a possibility that the input image isthe image of the detection target, and is calculated in a mannerdifferent from that of calculating the first reliability.
 6. Thereliability calculating apparatus according to claim 3, wherein theprogramming code further causes the processor to calculate a thirdreliability based on the first reliability and one or more secondreliabilities, and wherein each of the second reliabilities indicates apossibility that the input image is the image of the detection target,and is calculated in a manner different from that of calculating thefirst reliability.
 7. The reliability calculating apparatus according toclaim 1, wherein the programming code, in calculating the firstreliability, causes the processor to calculate the reliability on thebasis of the difference information and the calculated part likelihood.8. The reliability calculating apparatus according to claim 7, whereinthe programming code further causes the processor to calculate a thirdreliability based on the first reliability and one or more secondreliabilities, and wherein each of the second reliabilities indicates apossibility that the input image is the image of the detection target,and is calculated in a manner different from that of calculating thefirst reliability.
 9. The reliability calculating apparatus according toclaim 1, wherein the programming code further causes the processor tocalculate a third reliability based on the first reliability and one ormore second reliabilities, and wherein each of the second reliabilitiesindicates a possibility that the input image is the image of thedetection target, and is calculated in a manner different from that ofcalculating the first reliability.
 10. A reliability calculating methodfor detection of a target as a face performed by a computer, comprisingthe computer-performed steps of: retrieving, from a classifier storingdevice, information on a classifier which outputs a part likelihood of apredetermined part of a detection target calculated based on imagefeature values concerning the predetermined part of the detectiontarget; calculating, using data of an image region included in an inputimage, a part likelihood of the input image by using the retrievedinformation on the classifier; determining a position, in the inputimage, of the predetermined part of the detection target based on thecalculated part likelihood; retrieving information on a referenceposition of the predetermined part of the detection target stored in areference position information storing device; calculating differenceinformation between the reference position of the predetermined part ofthe detection target and the determined position of the predeterminedpart; and calculating a first reliability that indicates a possibilitythat the input image is an image showing the detection target based on afacial feature point likelihood calculated based on the image featurevalues, wherein the image feature values include a difference inintensity values, wherein, in calculating the difference informationbetween the reference position of the predetermined part of thedetection target and the determined position of the predetermined part,the difference information is calculated on the basis of whether theposition of the predetermined part in the input image determined by theposition determining unit is an outlier using a method of robustestimation, and wherein the detection target is a face.
 11. A programfor detection of a target as a face comprising computer-readablesoftware code stored on a non-transitory recording medium that, uponexecution by a processor device of a computer, causes, the computer toperform steps of: retrieving, from a classifier storing device,information on a classifier which outputs a part likelihood of apredetermined part of a detection target calculated based on imagefeature values concerning the predetermined part of the detectiontarget; calculating, using data of an image region included in an inputimage, a part likelihood of the input image by using the retrievedinformation on the classifier; determining a position of thepredetermined part in the input image based on the calculated partlikelihood; retrieving information on a reference position of thepredetermined part of the detection target stored in a referenceposition information storing device; calculating difference informationbetween the reference position of the predetermined part of thedetection target and the determined position of the predetermined part;and calculating first reliability that indicates a possibility that theinput image is an image showing the detection target based on a facialfeature point likelihood calculated based on the image feature values,wherein the image feature values include a difference in intensityvalues, wherein, in calculating the difference information between thereference position of the predetermined part of the detection target andthe determined position of the predetermined part, the differenceinformation is calculated on the basis of whether the position of thepredetermined part in the input image determined by the positiondetermining unit is an outlier using a method of robust estimation, andwherein the detection target is a face.